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Abstract 

We design algorithms for fitting a high-dimensional statistical model to a large, sparse net¬ 
work without revealing sensitive information of individual members. Given a sparse input graph 
G, our algorithms output a node-differentially-private nonparametric block model approxima¬ 
tion. By node-differentially-private, we mean that our output hides the insertion or removal of 
a vertex and all its adjacent edges. If G is an instance of the network obtained from a generative 
nonparametric model defined in terms of a graphon W, our model guarantees consistency, in 
the sense that as the number of vertices tends to infinity, the output of our algorithm converges 
to W in an appropriate version of the L2 norm. In particular, this means we can estimate the 
sizes of all multi-way cuts in G. 

Our results hold as long as W is bounded, the average degree of G grows at least like the 
log of the number of vertices, and the number of blocks goes to infinity at an appropriate rate. 
We give explicit error bounds in terms of the parameters of the model; in several settings, our 
bounds improve on or match known nonprivate results. 
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1 Introduction 


Differential Privacy. Social and communication networks have been the subject of intense study 
over the last few years. However, while these networks comprise a rich source of information for 
science, they also contain highly sensitive private information. What kinds of information can we 
release about these networks while preserving the privacy of their users? Simple measures, such 
as removing obvious identifiers, do not work; for example, several studies (e.g., [HI [52]) reidentified 
individuals in the graph of a social network even after all vertex and edge attributes were removed. 
Such attacks highlight the need for statistical and learning algorithms that provide rigorous privacy 
guarantees. 

Differential privacy [28], which emerged from a line of work started by [25], provides meaningful 
guarantees in the presence of arbitrary side information. In a traditional statistical data set, where 
each person corresponds to a single record (or row of a table), differential privacy guarantees that 
adding or removing any particular person’s data will not noticeably change the distribution on the 
analysis outcome. There is now a rich and deep literature on differentially private methodology for 
learning and other algorithmic tasks; see m for a recent tutorial. By contrast, differential privacy 
in the context of graph data is much less developed. There are two main variants of graph differential 
privacy: edge and node differential privacy. Intuitively, edge differential privacy ensures that an 
algorithm’s output does not reveal the inclusion or removal of a particular edge in the graph, while 
node differential privacy hides the inclusion or removal of a node together with all its adjacent edges. 
Edge privacy is a weaker notion (hence easier to achieve) and has been studied more extensively, 
with particular emphasis on the release of individual graph statistics [S3 |55l [391 ED S3 00], the 
degree distribution [32] EH ESI SHUT], and data structures for estimating the edge density of all 
cuts in a graph mm- Several authors designed edge-differentially private algorithms for fitting 
generative graph models [III 113 03 [37] Eo], but these do not appear to generalize to node privacy 
with meaningful accuracy guarantees. 

The stronger notion, node privacy, corresponds more closely to what was achieved in the case 
of traditional data sets, and to what one would want to protect an individual’s data: it ensures 
that no matter what an analyst observing the released information knows ahead of time , she learns 
the same things about an individual Alice regardless of whether Alice’s data are used or not. In 
particular, no assumptions are needed on the way the individuals’ data are generated (they need 
not even be independent). Node privacy was studied more recently @11 [22] 10, 53], with a focus on 
on the release of descriptive statistics (such as the number of triangles in a graph). Unfortunately, 
differential privacy’s stringency makes the design of accurate, node-private algorithms challenging. 

In this work, we provide the first algorithms for node-private inference of a high-dimensional 
statistical model that does not admit simple sufficient statistics. 

Modeling Large Graphs via Graphons. Traditionally, large graphs have been modeled using 
various parametric models, one of the most popular being the stochastic block model [[35]• Here 
one postulates that an observed graph was generated by first assigning vertices at random to one 
of k groups, and then connecting two vertices with a probability that depends on the groups the 
two vertices are members of. 

As the number of vertices of the graph in question grows, we do not expect the graph to be well 
described by a stochastic block model with a fixed number of blocks. In this paper we consider 
nonparametric models (where the number of parameters need not be fixed or even finite) given in 
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terms of a graphon. A graphon is a measurable, bounded function W : [0,1] 2 —>• [0, oo) such that 
W(x,y ) = W(y,x), which for convenience we take to be normalized: f W = 1. Given a graphon, 
we generate a graph on n vertices by first assigning i.i.d. uniform labels in [0,1] to the vertices, 
and then connecting vertices with labels x,y with probability p n W(x,y), where p n is a parameter 
determining the density of the generated graph G n with p n Halloo < 1. We call G n a W-random 
graph with target density p n (or simply a p n VF-random graph). 

To our knowledge, random graph models of the above form were first introduced under the name 
latent position graphs [533, and are special cases of a more general model of “inhomogeneous random 
graphs” defined in [12], which is the first place were n-dependent target densities p n were considered. 
For both dense graphs (whose target density does not depend on the number of vertices) and sparse 
graphs (those for which p n —> 0 as n —> oo), this model is related to the theory of convergent graph 
sequences For dense graphs it was first explicitly proposed in [26], though it can 

be implicitly traced back to [361 El, where models of this form appear as extremal points of two- 
dimensional exchangeable arrays; see [24] (roughly, their results relate graphons to exchangeable 
arrays the way de Finetti’s theorem relates i.i.d. distributions to exchangeable sequences). For 
sparse graphs, m offers a different nonparametric approach. 

Estimation and Identifiability. Assuming that G n is generated in this way, we are then faced 
with the task of estimating W from a single observation of a graph G n . To our knowledge, this task 
was first explicitly considered in [7j, which considered graphons describing stochastic block models 
with a fixed number of blocks. This was generalized to models with a growing number of blocks 
[56 . 23|, while the first estimation of the nonparametric model was proposed in [8]. Various other 
estimation methods were proposed recently, for example [351 [581 sa [591 [201 SI ED [30113 HU [H [2]- 
These works make various assumptions on the function W, the most common one being that after 
a measure-preserving transformation, the integral of W over one variable is a strictly monotone 
function of the other, corresponding to an asymptotically strictly monotone degree distribution of 
G n . (This assumption is quite restrictive: in particular, such results do not apply to graphons that 
represent block models.) For our purposes, the most relevant works are Wolfe and Olhede [59], Gao 
et al. ED], Chatterjee [21j and Abbe and Sandon [2j, which provide consistent estimators without 
monotonicity assumptions (see “Comparison to nonprivate bounds”, below). 

One issue that makes estimation of graphons challenging is identifiability. multiple graphons 
can lead to the same distribution on G n . Specifically, two graphons W and W lead to the same 
distribution on W-random graphs if and only if there are measure preserving maps 0,0 : [0,1] —> 
[0,1] such that W & = W where is defined by W(x,y) = W(</>(x), 0(y)) [23[ fl5]. Hence, there 
is no “canonical graphon” that an estimation procedure can output, but rather an equivalence 
class of graphons. Some of the literature circumvents identifiability by making strong additional 
assumptions, such as strict monotonicity, that imply the existence of canonical equivalent class 
representatives. We make no such assumptions, but instead define consistency in terms of a metric 
on these equivalence classes, rather than on graphons as functions. We use a variant of the L 2 
metric, 

S 2 (W,W') = inf WWt-W'h. (1) 

[ 0 , 1 ] >■ [ 0 , 1 ] 

where 0 ranges over measure-preserving bijections. 
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1.1 Our Contributions 

In this paper we construct an algorithm that produces an estimate W from a single instance G n 
of a kb-random graph with target density p n (or simply p, when n is clear from the context). We 
aim for several properties: 

1. W is differentially private; 

2. W is consistent, in the sense that S< 2 ( W, W) —>• 0 in probability as n —oo; 

3. W has a compact representation (in our case, as a matrix with o(n) entries); 

4. The procedure works for sparse graphs, that is, when the density p is small; 

5. On input G n , W can be calculated efficiently. 

Here we give an estimation procedure that obeys the first four properties, leaving the question 
of polynomial-time algorithms for future work. Given an input graph G n , a privacy-parameter e 
and a target number k of blocks, our algorithm A produces a £:-block graphon W = A(G n ) such 
that 


• A is e-differentially node private. The privacy guarantee holds for all inputs, independent of 
modeling assumptions. 

• Assume that (1) W is an arbitrary graphon, normalized so f W = 1; (2) the expected average 
degree (n — l)p grows at least as fast as logn; and (3) k goes to infinity sufficiently slowly with 
n. Then when G n is plb-random, the estimate W for W is consistent (that is, ^(kb, kb) —> 0, 
both in probability and almost surely). 


Combined with the general theory of convergent graphs sequences, these result in particular give a 
node-private procedure for estimating the edge density of all cuts in a plb-random graph, see (24) 
in Section [2.21 below. 

The main idea of our algorithm is to use the exponential mechanism of |50j to select a block 


model which approximately minimizes the £2 distance to the observed adjacency matrix of G, under 
the best possible assignment of nodes to blocks (this explicit search over assignments makes the 
algorithm take exponential time). In order to get an algorithm that is accurate on sparse graphs, 
we need several nontrivial extensions of current techniques. To achieve privacy, we use a new 
variation of the Lipschitz extension technique of HD [22] to reduce the sensitivity of the 62 distance. 
While those works used Lipschitz extensions for noise addition, we use of Lipshitz extensions inside 
the “exponential mechanism” [50] (to control the sensitivity of the score functions). To bound 
our algorithm’s error, we provide a new analysis of the .^-minimization algorithm; we show that 
approximate minimizers are not too far from the actual minimizer (a “stability” property). Both 
aspects of our work are enabled by restricting the ^-minimization to a set of block models whose 
density (in fact, L ^ norm) is not much larger than that of the underlying graph. The algorithm is 
presented in Section [3] 

Our most general result proves consistency for arbitrary graphons kb but does not provides 
a concrete rate of convergence. However, we provide explicit rates under various assumptions on 
kb. Specifically, we relate the error of our estimator to two natural error terms involving the 
graphon kb: the error (kb) of the best fc-block approximation to kb in the L 2 norm (see © 
below) and an error term e n (kb) measuring the L 2 -distance between the graphon kb and the matrix 
of probabilities H n {W) generating the graph G n (see @ below.) In terms of these error terms, 
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Theorem [Q shows 


5 2 (W,W) < ef\w) + 2e n {W) + 0 P \ 


I log k k 2 log n 1 

pn V ne pen 


(2) 


Along the way, we provide a novel analysis of a straightforward, nonprivate least-squares esti¬ 
mator, whose error bound has a better dependence on k: 


62 ( W, W aonprivate J < 


(O) 


(W) + 2e n (W) + Op | 


I log k k 2 

pn pn 2 


(3) 


It follows from the theory of graph convergence that for all graphons W, we have e[°' 1 (IF) — > 0 
as k —> 00 and e n (W) —> 0 almost surely as n —> 00 . As proven in Appendix |P] we also have 
£n{W) = Op{ef > \w) + yjk/n ), though this upper bound is loose in many cases. 

As a specific instantiation of these bounds, let us consider the case that W is exactly described 
by a fc-block model, in which case e| r °' ) (IF) = 0 and e n (W) = Op{fjk/n) (see Appendix D). For 
k < {n/ log 2 n) 1 / 3 , p > log {k)/k and constant e, our private estimator has an asymptotic error that 
is dominated by the (unavoidable) error of e n (W ) = fjkjn , showing that we do not lose anything 
due to privacy in this special case. Another special case is when W is a-Holder continuous, in 
which case ef > \w) = 0{k~ a ) and e n (W) = Op(n _a / 2 ); see Remark [ 2 ] below. 


Comparison to Previous Nonprivate Bounds. We provide the first consistency bounds for 
estimation of a nonparametric graph model subject to node differential privacy. Along the way, for 
sparse graphs, we provide more general consistency results than were previously known, regardless 
of privacy. In particular, to the best of our knowledge, no prior results give a consistent estimator 
for W that works for sparse graphs without any additional assumptions besides boundedness. 

When compared to results for nonprivate algorithms applied to graphons obeying additional 
assumptions, our bounds are often incomparable, and in other cases match the existing bounds. 

We start by considering graphons which are themselves step functions with a known number 
of steps k. In the dense case, the nonprivate algorithms of [30] and [21], as well as our nonprivate 
algorithm, give an asymptotic error that is dominated by the term e n {W) = 0(^/k/n), which is of 
the same order as our private estimator as long as k = o(n 1//3 ). [59] provided the first convergence 
results for estimating graphons in the sparse regime. Assuming that W is bounded above and 
below (so it takes values in a range [Ai,A 2 ] where Ai >0), they analyze an inefficient algorithm 
(the MLE). The bounds of [59J are incomparable to ours, though for the case of fe-block graphons, 
both their bounds and our nonprivate bound are dominated by the term f/k/n when p > (log k)/k 
and k < pn. A different sequence of works shows how to consistently estimate the underlying 
block model with a fixed number of blocks k in polynomial time for very sparse graphs (as for our 
non-private algorithm, the only thing which is needed is that np — > 00 ) GHEE] ; we are not aware 
of concrete bounds on the convergence rate. 

For the case of dense a-Holder-continuous graphons, the results of m give an error which is 
dominated by the term e n (W) = Op{n~ a / 2 ). For a < 1/2, our nonprivate bound matches this 
bound, while for a > 1/2 it is worse. [59] consider the sparse case. The rate of their estimator 
is incomparable to the rate of our estimator; further, their analysis requires a lower bound on the 
edge probabilities, while ours does not. 

See Appendix [A] for a more detailed discussion of the previous literature. 
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2 Preliminaries 


2.1 Notation 

For a graph G on [n] = {1,..., re}, we use E(G) and A(G) to denote the edge set and the adjacency 
matrix of G, respectively. The edge density p(G) is defined as the number of edges divided by Q). 
Finally the degree di of a vertex i in G is the number of edges containing i. We use the same notation 
for a weighted graph with nonnegative edge weights where now p(G) = n ^_-^ an d 

di = Ylj^i Pij- We use G n to denote the set of weighted graphs on n vertices with weights in [0,1], 
and G nt d to denote the set of all graphs in G n that have maximal degree at most d. 

From Matrices to Graphons. We define a graphon to be a bounded, measurable function 
W : [0, l] 2 —> M + such that W(x,y) = W(y,x ) for all x,y E [0,1]. It will be convenient to 
embed the set of a symmetric n x n matrix with nonnegative entries into graphons as follows: let 
V n = (Ji,... I n ) be the partition of [0,1] into adjacent intervals of lengths 1/n. Define IF [A] to be 
the step function which equals on I t X Ij. If A is the adjacency matrix of an unweighted graph 
G, we use W[G\ for IF [A], 

Distances. For p E [1, oo) we define the L p norm of an re X n matrix A by \\A\\ P = Aj Xre j I Ajj| p , 
and the L p norm of a (Borel)-measurable function IF : [0, l] 2 —> M by ||/||p = f \f(x, y)\ p dxdy. 
Associated with the L 2 -norm is a scalar product, defined as ( A , B) = A? Yli j A-ijBij for two re x n 
matrices A and B, and (U,W) = f U(x, y)W(x, y)dxdy for two square integrable functions U,W : 
[0, l] 2 —> M. Note that with this notation, the edge density and the Li norm are related by 
l|G||i = ^P(G). 

Recalling 0- we dehne the 82 distance between two matrices A, B, or between a matrix A and 
a graphon W by (52(A, B) = 52 {W[A\ 1 W[B\) and (52(A, W) = 82 (W [A], W). In addition, we will 
also use the in general larger distances ^(A, B) and ^(A, IF), defined by taking a minimum over 
matrices A' which are obtained from A by a relabelling of the indices: ^(A, B) = min^/ ||A 7 — B H 2 
and ^2 (A, IF) = min^/ 11 A 7 — W\\ 2 - 

2.2 IF-random graphs and graph convergence 

W-random graphs and stochastic block models. Given a graphon IF we define a random 
n X n matrix H n = H n [W ) by choosing n “positions” x±,... ,x n i.i.d. uniformly at random from 
[0,1] and then setting ( H n )ij = W(xi,Xj). If HlFHoo < 1, then H n (W) has entries in [0,1], and we 
can form a random graph G n = G n (W) on re-vertices by choosing an edge between two vertices 
i < j with probability ( H n )ij , independently for all i < j. Following [36] we call G n {W) a IF- 
random graph and H n (W) a IF-weighted random graph. We incorporate a target density p n (or 
simply p, when re is clear from the context) by normalizing IF so that J IF = 1 and taking G to 
be a sample from G n (pW). In other words, we set Q = H n (pW ) = pH n (W ) and then connect i to 
j with probability Qij, independently for all i < j. 

The error from our main estimates measuring the distance between H n (W ) and IF is defined 
as 

e n (W) = 8 2 (H n (W),W) (4) 

and goes to zero as re —> 00 by the following lemma, which follows easily from the results of di- 
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Lemma 1. Let W be a graphon with ||W’|| 00 < oo. With probability one, ||i7 n (VL)||i —> ||W||i and 
e n (W)-> 0. 

Stochastic block models are specific examples of tU-random graph in which W is constant on 
sets of the form Jj x Ij, where (/ 1 ,..., Ik) is a partition of [0,1] into intervals of possibly different 
lengths. 


Approximation by block models. In the opposite direction, we can map a function W to a 
matrix B by the following procedure. Starting from an arbitrary partition V = (Ifi,..., Yk) of [0,1] 
into sets of equal Lebesgues measure, define W/V to be the matrix obtained by averaging over sets 
of the form Yi xYj, 




1 

A (Yi)X{Yj) 



W(x, 


where A(-) denotes the Lebesgue measure. Finally, we will use W-p to denote the step function 


W V =Y J Yi X ly 7 . 

i,j£[k] 

Using the above averaging procedure, it is easy to see that any graphon W can be well approximated 
by a block model. Indeed, let 

e { °\w) = mm\\W-W[B]\\ 2 (5) 

B 

where the minimum goes over all k x k matrices B. Given that we are minimizing the ^-distance, 
the minimimizer can easily be calculated, and is equal to W-p k , where Vk is a partition of [0,1] into 
adjacent intervals of lengths 1/k. It then follows from the Lebesgue density theorem (see, e.g., jTTj 
for details) that e^\w) = \\W — W-p k ||2 —> 0 as k —> oo. 

We will take the above approximation as a benchmark for our approach, and consider it the 
error an “oracle” could obtain (hence the superscript O ). 


Convergence. The theory of graph convergence was first developed 031 ns mu, where it was 
formulated for dense graphs, and then generalized to sparse graphs in m m Eg. One of the 
notions of graph convergence considered in these papers is the notion of convergence in metric. 
The metric in question is similar to the metric 62, but instead of the L 2 -norm, one starts from the 
cut-norm || • ||□ first defined in [ 29] . 

\\W\\ n = sup f W , 

S,Tc[0,l] JSxT 

where the supremum goes over all measurable sets S, T C [0,1]. The cut-distance between two 
integrable functions U, W ; [0, l] 2 —> M is then defined as 

Sn(U,W) = inf||C/^- W\\ n , 

<t> 

where the inf goes over all measure preserving bijections on [0,1]. We will also need the following 
variations: a distance 6n(G,G') between two graphs on the same node set, as well as a distance 
5n(G,W) between a graph G and a graphon W, defined as da(G,G') = mine// ||IF[G"] — TU[G]||n 





and da(G,W) = mine-// HWfG"] — W||n, respectively, where the minimum goes over graphs G" 
isomorphic to G. 

Given these notions, we say a (random or deterministic) sequence G n of graphs converges to a 
graphon W in the cut metric if, as n —> oo, 

With this notion of convergence, for any graphon W with f W = 1, a sequence of W-random 
graphs G n with target density p n converges to the generating graphon W. This was shown for 
bounded W and n-independent target densities p with /3||W||oo < 1 in [TJ], but the statement is 
much more general, and in particular holds for arbitrary target densities p n as long as np n —> oo 
and limsupp n || W||oo < 1 [jlT j. 


Estimation of Multi-Way Cuts. Using the results of [18| . the convergence of G n in the cut- 
metric implies many interesting results for estimating various quantities defined on the graph 
G n . Indeed, a consistent approximation W to W in the metric 82 is clearly consistent in the weaker 
metric But this distance controls various quantities of interest to computer scientists, e.g., 
the size of all multi-way cuts, implying that a consistent estimator for W also gives consistent 
estimators for all multi-way cuts. 

To formalize this, we need some notation. Given a weighted graph G on [n] with node-weights 
one and edge-weights (3 xy (G ), and given a partition V = (Vi,..., V q ) of [n] into q groups, let G/V 
be the weighted graph with weights 


a i (G/V) = \V i \/\V(G)\ and fo^G/V) 


1 

n^Gh 


E MW- 

xeVi,y&Vj 


We call G/V a g-quotient or q- way cut of G, and denote the set of all q- way cuts by S q (G): 

S q (G) = {G/V : V is a partition of [n] into q sets } . 

We also consider the set of fractional q-way cuts, S q (G) = {G/p}, defined in terms of frac¬ 
tional q-partitions p. A fractional g-partition of V(G) is a map p : V{G) -> A ? : 1 4 p(x), 
where A q is the simplex A q = {p = (pf) 6 [0, l] 9 : YliPi = 1}j an d the corresponding frac¬ 
tional quotient G/p is the weighted graph with weights ai(G/p ) = iv(a)\ S* Pi( x ) an d Pij(G/p) = 

n 2 ||G||i ^2xeVi,y£Vj Pxy{G)pi(x)pj(x). 

The set of fractional (/-partitions of a graphon W, S g (W), is defined similarly: S q (W) = {W)p \ 
p : [0,1] —»• A q }, with a fractional partition now a measurable function p : [0,1] —> A q , and W/p 
given in terms of the weights 

a-iiW/p) = J pi(x)dx and Pij{W/p) = J Pi(x)pj(y)W(x,y). 


To measure the distance between the various sets of q -way cuts, we use the Hausdorff distance 
between sets S, S' C R q+q2 , 

d^ aus (S', S') = maxjsup inf \\H — H'W^, sup inf \\H — H'Wqq}, 
hgs h '^ s ' H'eS lH ^ s 
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where || • ||oo is the Loo-norm ||H — H'\\^ = maxjmaxj | oii(H) — cti{H')\, max ij \f3ij(H) — /3ij(H')\}. 

It was shown in m that if G n converges to W in the cut-metric, then d^jf us (S q (G n ), S q (W)) —> 
0. In particular, a consistent estimator W for the generating graphon W of a W- random graph G n 
leads to a consistent estimator S q (W) for the cuts S q (G n ), in the sense that d^’ us (S q (G n ), S q (W)) —> 
0. With a little more work, we can give quantitative bounds; see Theorem [3] below. 

2.3 Differential Privacy for Graphs 

The goal of this paper is the development of a differentially private algorithm for graphon estimation. 
The privacy guarantees are formulated for worst-case inputs — we do not assume that G is generated 
from a graphon when analyzing privacy. This ensures that the guarantee remains meaningful no 
matter what an analyst knows ahead of time about G. 

In this paper, we consider the notion of node privacy. We call two graphs G and G' node 
neighbors if one can be obtained from the other by removing one node and its adjacent edges. 

Definition 1 (e-node-privacy). A randomized algorithm A is e-node-private if for all events S in 
the output space of A, and node neighbors G,G', 

Pr[.4(G) G S] < exp(e) x Pr[„4(G") G S] . 

We also need the notion of the node-sensitivity of a function / : O n —> M, defined as maximum 
rnax^G' I/(G) — f{g') |, where the maximum goes over node-neighbors. This constant is often called 
the Lipshitz constant of / 

Finally, we need a lemma concerning the extension of functions / : G -> R to functions 
/ : G n —> M. We say a function on adjacency matrices is nondecreasing if adding an edge to the 
adjacency matrix does not increase the value of the function. 

Lemma 2 ( jl8l ITT] ). For every function f : G nt d —> M, there is an extension f : G n —> M of f with 
the same node-sensitivity as f. If f is a nondecreasing linear function of the adjacency matrix, then 
we can select f to be nondecreasing and computable in polynomial-time and so that f(G) < f(G) 
for all graphs G G O n . 

The lemma is proved in Appendix [B| 


3 Differentially Private Graphon Estimation 

3.1 Least-squares Estimation 

Given a graph as input generated by an unknown graphon W, our goal is to recover a block-model 
approximation to W. The basic nonprivate algorithm we emulate is least squares estimation, which 
outputs the k X k matrix B which is closest to the input adjacency matrix A in the distance 

<5 2 (-B, A) = min \\B W - A\\ 2 , 

7T 

where the minimum runs over all equipartitions n of [n] into k classes, i.e., over all maps ir : [n] —> [k] 
such that all classes have size as close to n/k as possible, i.e., such that ||vr _1 (i)| — n/k\ < 1 for all i, 
and B n is the n x n block-matrix with entries ( B n ) xy = B n ^ n ^ y y If A is the adjacency matrix of a 
graph G, we write 5 2 {B , G) instead of 5 2 (B, A). In the above notation, the basic algorithm we would 
want to emulate is then the algorithm which outputs the least square fit B = argmin B 5 2 (B,G), 
where the argmin runs over all symmetric k x k matrices B. 
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3.2 Towards a Private Algorithm 

Our main idea to turn the least square algorithm into a private algorithm is to use the so-called 
exponential mechanism of McSherry and Talwar m- Applied naively, we would therefore want to 
output a random k x k matrix B according to the probability distribution 

Pr(B = B) oc exp C5%(B, A)j , 

with C chosen small enough to guarantee differential privacy. It is a standard fact from the theory 
of differential privacy, that C should be at most e over twice the node-sensitivity of the “score 
function”, here •). But this value of C turns out to be too small to produce an output that is 

a good approximation to the least square estimator. Indeed, for a given matrix B and equipartition 
7 r, the distance ||G — B^W^ can change by as much as A when G is replaced by a node-neighbor, 
regardless of the magnitude of the entries of B. To obtain differential privacy, we then would need 
to choose C > ne/2, which turns out to not produce useful results when the input graph G is 

sparse, since small values of C will lead to large errors relative to the least square estimator. 

To address this, we first note that we can work with an equivalent score that is much less 
sensitive. Given B and 7r, we subtract off the squared norm of G to obtain the following: 

score{B, 7r; G) = ||G||| - ||G - B v \\l = 2(G, B v ) - \\B^\\ 2 , and (6) 

score(B\G ) = max score(B, 7 r; G), (7) 

7T 

where the max ranges over equipartitions 7r : [n] —> [k]. For a fixed input graph G, maximizing the 
score is the same as minimizing the distance, i.e. argmin B G) = argmax B score(B; G). The 
sensitivity of the new score is then bounded by A • H-BHoo times the maximum degree in G (since 
G only affects the score via the inner product (G, B n )). But this is still problematic since, a priori, 
we have no control over either the size of ||-B||oo ° r the maximal degree of G. 

To keep the sensitivity low, we make two modifications: first, we only optimize over matrices 
B whose entries are of order p n (in the end, we expect that a good estimator will have entries 
which are not much larger than ||/OnbF||oo, which is of order p n ), and second we restrict ourselves to 
graphs G whose maximum degree is not much larger than one would expect for graphs generated 
from a bounded graphon, namely a constant times the average degree. While the first restriction is 
something we can just implement in our algorithm, unfortunately the second is something we have 
no control over: We need to choose C small enough to guarantee privacy for all input graphs, and 
we have set out to guarantee privacy in the worst case, which includes graphs with maximal degree 
n — 1. Here, we employ an idea from mm- we first consider the restriction of score{B,ir ; •) to 
G ni d n where d n will be chosen to be of the order of the average degree of G , and then extend it 
back to all graphs while keeping the sensitivity low. 

3.3 Private Estimation Algorithm 

After these motivations, we are now ready to define our algorithm. It takes as input the privacy 
parameter e, the graph G, a number k of blocks, and a constant A > 1 that will have to be chosen 
large enough to guarantee consistency of the algorithm. It outputs a matrix B from the set of 
matrices 

= {B G [0,p] kxk : all entries B tJ are multiples of A}. 
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Inside our algorithm, we use an e/2-private algorithm to get an estimate p for the edge density of 
G. We do so by setting p = p(G) + Lap(4/ne), where Lap(«) is a Laplace random variable with 
density h(z) = T e -M/ K . The existence of the Lipschitz extension used in the algorithm follows 
from Lemma [21 


Algorithm 1: Private Estimation Algorithm 
Input: e > 0, A > 1, an integer k and graph G on n vertices. 

Output: k x k block graphon (represented as a k X k matrix) estimating pW 

1 Compute an (e/2)-node-private density approximation p = p(G) + Lap(4/ne) ; 

2 d = Xpn (the target maximum degree) ; 

3 p = \p (the target L^ norm for the matrix B ) ; 

4 For each B and 7r, let score(S,7r; •) denote a nondecreasing Lipschitz extension of 
score(B , 7r; •) from G ny d to G n such that for all matrices A, scof e(B, i r; A ) < score(B, i r; A), 
and define 

score(i?; A) = maxscore(.B, 7r; A) 

7r 

5 return B, sampled from the distribution 


Pr (B = B) oc exp ^score (B; A 


where B ranges over matrices in B fl and A 



4A 2 p 2 
n ’ 


Lemma 3. Algorithm^ is e-node private. 

Proof. By Lemma [To| from Appendix [Bj the estimate p is e/2-private, so we want to prove that 
the exponential mechanism itself is e/2-private as well. In view of Lemma 11 from Appendix [Bj 
all we need to show is that the the vertex sensitivity of scor e(B; •) is at most A. To this end, we 
first bound the vertex sensitivity of the original score when restricted to graphs with degree d. Let 
G, G' E G n ,d be node neighbors. From ([6]), we see that 

2 _^ 

score(B, vr; G) - score{B, vr; G') = ^ (A xy - A' xy )B^ x )^ , 

s,y€[n\ 


7l~ 


where A, A! are the adjacency matrices of G and G'. Since A and A' differ in at most 2d entries, the 
score differs by at most ^dWBnWoo/n * 2 . This is at most A, since B E B^. Since score is a Lipschitz 
extension of score, the vertex sensitivity of score (over all neighboring graphs) is at most A, as 
required. □ 


Theorem 1 (Performance of the Private Algorithm). Let W : [0, l] 2 — > [0,A] be a normalized 
graphon, let 0 < pA < 1. let G = G n (pW), A > 1. and k be an integer. Assume that pn > 61ogn 
and 8A < A < \fn, 2 < k < minjny^f, e^~}. Then the Algorithm jfj outputs an approximation 
(p,B ) such that 


s 2 (w,- 

P 


■fwm) 


<e ( °\w) + 2e n {W) + 0 P \ 


' A 2 log k 
pn 


+ A 


k 2 log 


n 


+ 


ne 


X_ 

npe 
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The theorem will be proven in Section [5j 

In the course of the proof, we will prove results on the performance of a non-private algorithm, 
which is a variant of the standard least square algorithm, the main difference being that instead 
of minimizing 52 {B,A) over all matrices B, we only optimize it over matrices whose entries are 
bounded by a constant times the density of G. 


Algorithm 2: Nonprivate Algorithm 
Input: A > 1, an integer k and graph G on n vertices. 

Output: k x k block graphon (represented as a k x k matrix B ) estimating pW 

1 p <— A p(G) (the target Loo norm for the matrix B) ; 

2 return B G argmin 82 (R; G). 


Theorem 2 (Performance of the Nonprivate Algorithm). Let W : [0, l] 2 —>• [0, A] be a normalized 
graphon, let 0 < pA < 1, let G = G n (pW), A > 1, and k be an integer. If B is the least-squares 


estimator (Algorithm 


2 ), 2A < A < \Jn, 2 < k < min{n, /§, e 2 } 7 then 






In particular, 62 (w, ^j)IT[R]j —> 0 in probability if k —> 00 and A 2 ^^|^ + -» 0. 

Theorem [2] is proven in Section [4j 

Remark 1. While Theorem [7] and Theoremare stated in term of bounds which hold in probability, 
our proofs give slightly more, and allow us in particular to prove statements which hold almost 
surely as n —> 00 . Namely, they show that under the assumptions of Theorem^ the output B of 
the nonprivate algorithm is such that 

s i w -m w[ts] ) ief)(w)+ 0 {i^W^W] + ° (1)i 

they also show that if we replace the assumption np > 61ogn in Theorem^by the stronger assump¬ 
tion npe/ log n —> 00 , then the output B of the private algorithm is such that 


*0* \ w ^) <- ^ 

where in both expressions, o(l) is a term which goes to zero with probability one as n —>• 00 . 

Thus for both algorithm, as long as k grows sufficiently slowly with n, with probability one, the 
asymptotic error is of the form e^\w) + o(l), which is best possible, since we can’t do better than 
the best oracle block model approximation. 


Remark 2. Under additional assumptions on the graphon W, we can say a little more. For 
example, if we assume that W is Holder continuous, i.e, if we assume that the exists constants 
a G (0,1] and C < 00 such that \ W(x, y) — W(x', y')\ < CS a whenever \x — x'\ + |y — y'\ < 5, then 
we have that ef\w) = 0(k~ a ) and e n (W) = Op(n ~ a / 2 ). See Appendix [a] for details. 
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Theorems [l] and [ 2 ] imply that the sets of fractional q- way cuts of the estimator B from these 
theorems provide good approximations to the q- way cuts of the graph G (as defined in Section 2.2). 
Specifically: 

Theorem 3. Let q > 2 be an integer. 

(i) Under the assumptions of Theorem [1| 


d“ aus (S q (G), S q (B nonprivate )) = O p I ef ] \W) + e n (W) + A 2 
(ii) Under the assumptions of Theorem [IJ 


/logt 


d* aus (S q (G ), S q (B private )) = O p ( e^\W) + e n (W) + 


(O), 


'A 2 log k 
pn 


+ A 


k 2 log n A 
ne npe 


The proof of the theorem relies on the theory of graph convergence, in particular the results of 
PH El ESI, and is given in Appendix [Fj 

Remark 3. When considering the “best v block model approximation to W, one might want to 
consider block models with unequal block sizes; in a similar way, one might want to construct a 
private algorithm that outputs a block model with unequal size blocks, and produces a bound in 
terms of this best block model approximation instead of e^\w). With more cumbersome notation, 
this can be easily proved with our methods, with the minimal block size taking the role ofl/k in all 
our proofs. We leave the details to a journal version. 


4 Estimation Error of the Least Square Algorithm 

At a high level, our proofs of Theorems [l] and of [2] follow from the fact that for all B and 7 r, the 
expected score E[Score(B, n; G)\ is equal to the score Score(B,Tr,Q), combined with a concentra¬ 
tion argument. As a consequence, the maximizer B of Score(B]G ) will approximately minimize 
the L 2 -distance ^{B, Q), which in turn will approximately minimize ||^IT[-B] — W || 2 , thus relating 

the L 2 -error of our estimator B to the “oracle error” e^\w) defined in <§• 

In this section we present the analysis of exact and approximate least squares. This allows us 
to analyze the nonprivate algorithm. The analysis of the private algorithm (Theorem [lj requires 
additional arguments relating the private approximate maximizer to the nonprivate one; we present 
these in Section [ 5 ]). 

Our main concentration statement is contained in the following proposition, which we prove 
in Section |4.1| below. To state it, we define, for every symmetric n x n matrix Q with vanishing 
diagonal, Berno(Q) to be the distribution over symmetric matrices A with zero diagonal such that 
the entries {A^: i < j} are independent Bernouilli random variables with E Aij = Qij. 

Proposition 1. Let p > 0, Q E [0, l] nxn be a symmetric matrix with vanishing diagonal, and 
A ~ Berno(Q). If 2 < k < mining p(Q), e^( < 3) ri } and B £ is such that 

ScorefB ; A) > max ScoreiB ; A) — v 2 
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for some u > 0, then with probability at least 1 — 2e 


62 {B, Q ) < min <5 2 (R, Q) + v + O | 


l h 2 p(Q) 


k 2 log k ' 


+ 


n- 


n 


( 8 ) 


and in particular 


|S||2 < (2IIQII2 + 0 ( 1 + “) +o[fj^p(Q){^ + ^ 

2 k\ 


(9) 


< (2||Q||2 + n)( 1 + — J + 0(^v W*( 


Morally, the proposition contains almost all that is needed to establish the bound ([ 3 ]) proving 
consistency of the standard least squares algorithm (which, in fact, only involves the case v = 0), 


even though there are several additional steps needed to complete the proof (see Sections 4.2 and 


4.3 below). 

The proposition also contains an extra ingredient which is a crucial input for the analysis of 
the private algorithm: it states that if instead of an optimal, least square estimator, we output 
an estimator whose score is only approximately maximal, then the excess error introduced by the 
approximation is small. To apply the proposition, we then establish a a lemma which gives us a 
lower bound on the score of the output B in terms of the maximal score and an excess error v. 

There are several steps needed to execute this strategy, the most important ones involving a 
rigorous control of the error introduced by the Lipschitz extension inside the exponential algorithm 
(which in turn requires estimating the deviation of the maximal degree from the expected degree, 
a step where the condition that pn has to grow like log n is needed). The excess error v eventually 
turns into the second to last error term in while the difference between our private estimator 
p for the edge density and the actual edge density of G is responsible for the last one. 

The analysis of the private algorithm is presented in Section [5j the remainder of this section 
presents the detailed analysis of the least squares estimator. 

Remark 4. Note that for both the non-private algorithm and the private algorithm, the above 
proposition naturally gives a bound for the L 2 estimation error for matrix of probabilities Q. In 
fact, our proofs provide error bounds on j 2 (R,<5) which differ from ©>> (§ and the bounds in 
Theorem 3 in that (i) the error term 2 e n (W) is absent, and (ii) the oracle error e^\w) is replaced 
by an oracle error (H n ) for H n , see Theorems 4 , 5 and[h| Converting these bounds into bounds 
on <5 2 (W, jW[B]) and expressing the result in terms of e^\w) instead of ej^\H n ) then introduces 
the error term 2 e n (W) in (©, Q and the bounds in Theorem g 


4.1 Expectation and Concentration of Scores 

The following two lemmas contain the core of the argument outlined at the beginning of this section. 

Lemma 4 (Expected scores). Let Q 6 [0, l] nxn be a symmetric matrix with vanishing diagonal, let 
A ~ Berno(Q), and let B,B' be k x k matrices. Then 

B) — S 2 (Q, B') = maxE[5core(R / , G)] — maxE[5core(R, 7 r; G)], 

n' 7T 

where the two max's go over equipartitions : [n] —> [k]. 
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Proof. By linearity of expectation, we have 


E score(B, i r; A) = E(2(^4, B n ) — ||i?, r || 2 ) = 2 (Q, B n ) — 11^ 7 ,-111 
= \\Q\\ 2 2 -\\Q-B n \\ 2 2 . 


Taking into account the dehnition of <52 (B, Q ), the lemma follows. □ 

Our second lemma states that the realized scores are close to their expected values. The proof 
is based on a careful application of the concentration bounds. The argument is delicate because we 
must take advantage of the low density (when p is small). 

Lemma 5 (Concentration of scores). Let p > 0, let Q £ [0, l] raxn be a symmetric matrix with 
vanishing diagonal and let A ~ Berno(Q). If 2 < k < minjny 7 p(Q), e p ^ n }, then, with probability 
at least 1 — 2 e~ n 


log k\ 

n J 

for all equipartitions ir and all B £ [ 0,p] kxk . 

Proof. First, consider a specific pair B,n. Recall that 


| score(B, 7 r; ^4) — E [score(B, 1 r; A)]| = 0 \ p 


' l - 2 

! p(Q)[-2 




score(B, 1 r; A) — E [score(B, 1 r; ^4)] = 2{A — Q , B n ). 

We wish to bound the deviation of score(B,n; A) from its mean. Set p(Q) = p. The quantity 
S = jp - (A, B n ) = Ylicj B7r(l ^ 7r(j) Aij is a sum of Q) independent random variables in [0,1] with 
expectation E S < pQ). Using a slight variation on the standard Chernoff bound, which we state 
in Lemma [T7| we will bound the probability that S deviates from its mean by at most f3po, where 
po > EfS) will be chosen in a moment. Setting r/ = 2e~ n and 

k 2 + nlogA: + log(2/?y) / k 2 logfc 

3 pn 2 l V pn 2 pn 



the assumption k < mm{ny/p(Q)n, e p ^ n } implies f3 < 1, and setting po = 9 n 2 p, the bound from 
Lemma [TT] becomes 

2e -3/3 2 pn 2 = e -k 2 k -n v < ^ k~ n T), 

implying that 

Pr(|2M-Q,B.>|>^/J w )< 

Finally, we observe that for any A, the maximum of |(Q — A,B X )\ over all B £ [0 , p] kxk is the 
same as the maximum over all B £ {0, p} kxk . Taking a union bound over the (at most 2 k ~ k n ) pairs 

B, 7 r and observing that ^(3po = Oi p < p(^ + J ), we get the statement of the lemma. □ 
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Proof of Proposition^ 1} Let B £ be as specified, let B' S arbitrary, and let it, it' : n —> k be 
two equipartitions. By Lemmas [4] and [5j 

<5f (Q, B ) — <5|(Q. B') = max E [Score{B', i r x ; G)] — maxE[Score(B, 7 r; G)] 

ir' 7r 

< Score(B'-, G) — Score(B ; G) + O 

which implies the bound ([8]). (Taking square roots works since 

Gj > 0.) To prove we use that for an arbitrary equipartition 7r | 

11 B n 11 2 < HQH 2 + \\B n — Q H 2 - Inserting the definition of ^(B, Q ) and using the main statement plus 
the assumptions 2 <k< minjn-y/ p{Q), e p ^ n }, we obtain ([ 9 ]). .Q 



4.2 Estimation of the edge-probability matrix Q 

Up to technical details, Proposition [l] contains all that is needed to prove consistency of the least 
square algorithm. As indicated in Remark[4| we will first prove that the algorithm gives a consistent 
estimator for the matrix Q, and then use this prove that the output also gives a consistent estimator 
for W. The first statement is formalized in the following theorem. 


Theorem 4. Under the assumptions of Theorem ^ 

where 

i < f > \H)=MS 2 (B,H), 

JD 

with the inf going over all symmetric k x k matrices B. Moreover, a.s. as n —>• 00 , 


( 10 ) 


< 5 2 



B, H n (W) 


<e { °\H n (W)) + 0 



+ o(l). 


Proof. As a first step, we will bound the left hand side of (10) by conditioning on the event that 


2 < P(Q) < 2 P- 


( 11 ) 


By a concentration 
Lemma 12 (part 3) 


argument very similar to the proof of Lemma [ 5 ] above (in fact, it is easier, see 
in Appendix [c]) , we have that, with probability at least 1 — 2e~ n , 


P(G) 

p{Q) 


1 + 0 



1 + 0 
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We can now apply Proposition fll with v = 0 (since the nonprivate algorithm returns an exact 
minimizer). Recall that H n (W) = ^ and p = A p(G) = @(A p). We get that, with probability at 
least 1 — 4e _Tl , 


5 2 (^B,H n (W)^j < min <5 2 Qr, H n (W)j +0 



( 12 ) 


In the remainder of the proof, we bound the first term on the left-hand side above by relating it 
to the “oracle error” i^\H n (W)). Let B' and 7r be such that e^\H n {W)) = \\H n (W) — R^.|| 2 . It 
is easy to see that then B'^ is obtained from H n (W) by averaging over the classes of 7r, which in turn 
implies that = UR'IU < ||R n (IT)||oo < Halloo < A and p;|| 2 < \\H n (W )|| 2 = p^WQ || 2 . 

Define B by rounding all entries of p{G)B' down to the nearest multiple of 1/n, adding a rounding 
error of at most 1/n, so that \\B — p(G)B'\\ 00 < 1/n. Note that B' is on the scale of W and H n (W) 
(that is, we expect \\B'\\ = 0(1)), while B is on the scale of pW and Q ; hence, ||S 7r || 2 < ^^||Q|| 2 . 
Now p > p(G)A , pHoo < p(G)\\B'\\ 00 < p(G) A, and A < A/2. Thus, B is in the set B / that the 


algorithm searches over. We can bound the Hrst term in the left-hand side of (12) by 


S 2 (H n (W),-B 
P 


< 

< 


H n (W) — -B 
P 

H n [W) - ' 


P(G) 


B* 


+ 


- -B, 
p{G) p 


< A 0 ) (H n (W)) + 

< 4 0 ) {H n (W)) + 


Bi 1 B n 

+ 

-7- Bjr - — Bjr 

n P(G) 

2 

p(G) P 


np(G) 


p(G) 

WQh 

P 

p 


Combined with our previous two bounds and the fact that by we can bound ||Q|| 2 by ||Q|| 2 < 
\/||Q||i||<3||oo< \fp(QWp < V 2 Ap < a/A p, this implies that 


8 2 (jB,H n {W)j <ef\H n (W)) + V\ 


< ef\H n (W)) + VX 


1 - 


1 - 


p(Q) 


p 

p(Q) 


p 



We can now use ([9]) to bound ||R|| 2 by 0(||Q|| 2 + yjpp(Q)) = 0{\fXp) and thus 5 2 (R/ p, B / p(G)) 
by 0(\/A)|l — p(Q)/p | plus an error which can be absorbed into the error term above. We obtain 
that, conditioned on with probability at least 1 — 4e ", we have 


5 2 


P{G) 


B, H n (W) ) < ef ] {H n {W)) + 0 1 {/ A 2 | 


1 _ p(Q) 4 + Jf_ + log k \ 

p n 2 p np ) 


( 13 ) 


By Lemma 12 from Appendix [cj | p{Q) — p \ 2 = Op(Xp 2 /n), implying that 


i-m 


P 


n z 


n 


pn 
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On the other hand, again by Lemma 12 from Appendix [Cj the probability that ( p~T| ) does not hold 
is 0(X/n), showing that with probability 1 — 4e~ n — 0(\/n) = 1 — 0(A/n), 


-L.B,H n (W)) <e[°\H n (W)) + 0 P \ 


n£ + - 


np 


This holds conditioned on an event E of probability 0(A/n). To bound the contribution of E to 
the overall error, we bound ^y||-B||2 < ^)||i?||oo < A and ||iLn(bL)||2 < ||#n(bL)||oc < A, giving 

an error contribution of Op(A 2 /n) = Op(f /\ 2 / pn) which we can absorb into the error already 
present. 

To prove the almost sure statement, we use that e n (W) —> 0 almost surely, which by Lemma 12 
(part 2) from Appendix [C] implies that p{Q)/p —> 1 almost surely. Since the error probability in 
© is exponentially small, we can use the Borel-Cantelli Lemma to obtain the a.s. statement. □ 


4.3 Estimation of the graphon W 


To deduce Theorem 2 from Theorem 


we will bound i^\H n (W)) in terms of e^\w), and 


5 2 (B/p(G), W) in terms of 5 2 (B/p(G ), H n (W)). We will show that the leading error in both cases 
is an additive error of e n (W). To do this, we need two lemmas. 


Lemma 6. Fix n and k <n. 

(i) For each equipartition 7r : [n] 
tion. 


[k] and each permutation a : [re] —> [n], 7r o a is an equiparti- 


(ii) For all equipartitions : [n] —> [&:] there exists a permutation a : [n] — > [n] such that 

n' = IT O ( 7 . 

Proof. Any equipartition must have exactly L- classes of size [n/k\ and L+ classes of size \n/k~\, 
where L± are determined by the equations L_ + L + = k 1 L_[n/feJ + L + \n/k~\ = n; and any 
partition with these properties is an equipartition. The statement follows. □ 

To state the next lemma, we define the standard equipartition n of [n] into k classes to be the 
partition into the classes Ii = + 1,... ,nj}, i £ [&], where n* = \in/k\. Note that no = 0, 

nk = n, and \n/k\ < \ni + \ — nf\ < \n/k~\. 

Lemma 7. Let B be a symmetric k X k matrix with nonnegative entries, and let it be the standard 
equipartition of [n] into k classes. Then 


|| W[B\-W[B„ 


I 2 < 


dVil, 

n 


Proof. Let I\,... ,Ik be adjacent intervals of length 1/k, and for i = 1,..., k, let Ji be the set of 
point in x G such that x < ni/n, and let A,; = I, \ Jp Then (VL[L>] — W[B n ])(x,y) = 0 unless 
(x, y) lies in one of the 3fc 2 sets = A* x A j , = E \ Aj x A j or R.ff = A, x Ij \ A j , i,j 6 [k]. 


Taking, e.g., (x,y) e we have that |(Ty[5]-W[B 7r ])(x,y)| 2 = \Bij-B i+ ij +l \ 2 < B^+Bf^ +l 
(note that the set A k is empty, so that here we only have to consider i,j < k — 1). In a similar 
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way, the difference in is bounded by B 2 - + Bf +1 and the difference in is bounded by 
B 2 - + B 2 - +1 . The total contribution of all these sets can then be bounded by 


^ B ^nk 

v 


n 2 ) + H B W n k 


+ 3 B i+l,j+l T2 ^ “ 


V 


ij 


ri¬ 


nk 






4 k 
n 


B 


2 

2- 


Q 

Proof of Theorem^ We start by bounding 52 (B/p(G),W). Let n : [n] —> [k] be a standard 
equipartition, and let (I\,, I n ) be a partition of [0,1] into adjacent intervals of lengths 1/n. By 
the triangle inequality, the fact that the set of measure preserving bijections n : [0,1] —> [0,1] 
contains all bijections which just permute the intervals I\,... ,I n and Lemma [6] 

5 2 - } B,W^j < -L.S 2 (W[B], W[B n ]) + 5 2 (j^W[B],W[H n (W)]') + 5 2 {W[H n {W)], W) 

< II' W[B\ - W[B n \|| 2 + <5 2 (j^B, H n (W)j + S 2 (H n (W ), W). 

The third term is equal to e n (W). To bound the first term, we first condition on the event 0. 
and then use ([9]) together with Lemma [7] to conclude that conditioned on ©, with probability at 
least 1 — 4e -n , 

In view of Lemma [12] from Appendix [Cj the probability that this bound does not hold is bounded 
by 4e~ n + 0(X/n) = 0(X/n), so in view of the fact that B e £> /i; which shows that \\B\\ 2 /p(G) < A, 

we see that the contribution of the failure event is again bounded by Op(X 2 /n) = O^X 2 -jf-^j = 
O(^). All together, this proves that 


The corresponding a.s. bound follows again from the fact that p(Q) —> p a.s., and the fact that all 
other failure probabilities are exponentially small. 

Next fix B such that it is a minimizer in <©• That implies that B is obtained from W by 
averaging over a partition of W into k classes, which in particular implies that ||i?|| 2 < ||W|| 2 < 
y/WWWooWWlh < \/X Together with Lemma M this implies that there is an equipartition vr : [n] —> 
[k] such that 

ef\w) > \\W-W[B}\\ 2 

> \\W-W[B„]\\ 2 -J^n 

I Ah 

> 6 2 (B n ,W) - —n 
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Using Lemma [H] to express 52 (B,H n (W)) as a minimum over permutations a : [n] —> [n], we then 
bound 

4 ° ] (W) < 5 2 (B,H n {W )) = min || B % - [H n (W)] a || 2 

a 

< ||W[S ff ] - W\\ 2 + S 2 (H n (W), W) 

<ef\w) + e n {W) + ^n, 


where in the first line we use [H n (W)] a to denote the matrix with entries [H n {W)\ a ^ a ^ y y Together 
with (14) this completes the proof of the theorem. □ 


5 Analysis of the Private Algorithm 

In this section we prove consistency of the private algorithms. Our analysis relies on some basic 
results on differentially private algorithms from previous work, which are collected in Appendix [B| 
Compared to the analysis of the non-private algorithms, we need to control several additional 
error sources which were not present for the nonprivate algorithm. In particular, we will have 
to control the error between p and p(G), the fact that the algorithm (approximately) maximizes 
scor e(B;G) instead of Score(B] Q), and the error introduced by the exponential sampling error. 
The necessary bounds are given by the following lemma. To state it, we denote the maximal degree 
in G by d max (G). 

Lemma 8. Let (p, B) be the output of the randomized Algorithm ^! j Then the following properties 
hold with probability at least 1 — 2 e~ npe ^ ie with respect to the coin flips of the algorithm: 

1 ) \p{G) - p\ < p/4. 

2) If d max (G) < \p/A and p{G) > p/2, then 

Score(B; G) > max Score{B; G) - + D l°gn 

seS M ne 

Proof. Observing that Pr{|Lap(4/ne)[ > x} = exp(— xne/A), we get that 

Pr (|p(G) - p\> dp) = e~ 5npe / 4 , (15) 


which immediately gives (1). 

To prove (2), we first use (1) and the assumptions on p(G) and d ma , x (G) to bound 

A p > A (p(G) - p/4) > Ap/4 > d max (G). 

This implies that the extended score is equal to the original score. 

We conclude the proof by using Lemma 11 to show that with probability at least 1 — e~ n > 
1 — e - n P e / 16 5 the exponential mechanism returns a matrix B such that 


Score(B; G) > max Score(B; G) — 


4Alog(|^|) 


where A = A = 


_ a _ Adfi _ 4A 2 /3 2 


. Bounding |0^| < n fc2 , this completes the proof of the lemma. 


□ 
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Theorem [l] will follow from the following theorem in the same way as Theorem [2] followed from 
Theorem 01 


Theorem 5. Under the assumptions of Theorem 


fo[-B,H r 
-P 


<ef\H n {W)) + 0 P A 


pn 


ne npe 


(16) 


Moreover, if we replace the assumption np > 6 log n in Theorem [7] by the stronger assumption 
npe/ log n —> oo, then a.s. as n —> oo, 


4(1 + + +0(1 ). 

Proof of Theorem [5| With probability at least 1 — e _npe / 16 , we may assume that the output of the 
private algorithm obeys the conclusions of Lemma [8j With a decrement in probability of at most 
P n = 0(A/n), we then have that 


7 , < p{G) <2 p, | < p(Q) < 2p and ^ < p < 3 p. 


(17) 


Next use the assumption pn > 61ogn, the fact that 1 < A < A/8, and Lemma [l3| from Appendix [C] 
with (3 = A/(8A) to show that at a decrement in probability of at most e l ° sn ~ 2 i xpn < e~±8 Xpn , the 
maximal degree in G is at most (A + |)p < /(L Lemma [i] then allows us to use Proposition [l] with 


^ 16A 2 p 2 (£i 2 + 1) logn q(\ /fc 2 logn 


ne 


ne 


This introduces an additional error term / = O into the bound (12) and an extra 

error term of order O ^Apy ^ 2 ,j° s '' ^ in the upper bound ([£]), leading to the estimate that, with 
probability at least 1 — 4e -n — P n — e~^s xpn — e~ npe ! 16 = 1 — 0(A/n) — e ~ n ( npe \ 


and 


<M l -B,H n (W)^j < min 8 2 {jB,H n {W) ) +0( (/A 2 ( ^ 


|-B|| 2 = O (>/Ap + Ap 


+ —)+ At/^ 

y \n z p np J V ne 


n 


k 2 log n 


ne 


From here on we proceed as in the proof of (13), except that we now move from the minimizer B’ 
for ef P {H n {W)) to a matrix B £ Bf by rounding the entries of pB’ down to the nearest multiple 


of 1/n. Instead of (13), we now obtain the bound 


* ( \b, H n (W)) < if\H n{ W)) + O ( (/v (4- + + A, /k2 logn 


P 


n-p np 


+ Q V A 


ne 


P --l 


(18) 
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a bound which is valid with probability at least 1 — 4e n — P n — e ±& Xpn — e npe / 16 . Now the fact 
that |Lap(4/ne)| = Op(^) and p(G) = p( 1 + O p (\J A/n)) implies that 


1 - P - 





Combining this with (18), we obtain that with probability at least 1 — 4e n — P n — e 48 Xpn — e npe / 1G i 


1 


<52 ( j;B,H n (W) ) < ef\H n (W)) + Op 



k 2 log n V\ 

ne npe 


— • ( 19 ) 


The contribution of the failure event can now be bounded by 

Op (A (e~ n + - + e ~ npx/A8 + e~ npe/16 ) ) =O p (—+— 
\ \ n J J \n npe 


( 20 ) 


To complete the proof of the bound in probability, we have to add the error terms from (19) and 
20[). We can simplify the resulting expression somewhat by first noting that the left hand side of 


Eq. ([lp is of order at most A, which shows that for the bound on 82 not to be vacuous, we need 
— < k logTl < 1. We can therefore drop the term inside the fourth root of (|19|). Furthermore, 

n. — rtf — pn z ' 


by the assumption of the Theorem, A < y/n, which shows that the first term in (20) is 0( y / A 2 /n) 
and can hence be absorbed into the error terms in (19). This gives us the main theorem statement. 
To prove bounds which hold a.s., we note that for npe/ log n —> 00 , the error probability in 


(15) is summable (that is, the probability of error p n satisfies Yl^LiPn < °°) f° r a U <5 > 0, which 
together with our previous results implies that p/p ^ 1 with probability one. Since the probability 


of failure for all other events necessary for (19) to hold is summable as well, we get that a.s. 


<52 ( \b, H n (W) ) < ef ] (H n (W)) + O \ 


A 2 log k 
pn 


+ A 


k 2 logn^ \/A 
ne npe 


+ °( 1 )> 


where again o(l) is a term which goes to zero with probability one as n —> 00 . □ 

Proof of Theorem [7} The proof of Theorem [l] follows from Theorem [5] in essentially same way as 
Theorem [2] followed from Theorem [4} The only modification needed is that we now have to bound 
IS 2 (W[B\,W[B n ]) instead of -^^52{W[B\, W[B n ]). But this is even easier, since here we won’t 

B E Bfj, implies H-BHoo < A/5. With the 
a term which can be incorporated into 

the error term o(\\J k ~ n j. Et; 


need to distinguish several cases. Instead, we just use that 
help of Lemma |t| 


we then bound this error term by A \ / —, 

J \l n ’ 
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A Comparison to Nonprivate Bounds from Previous Work 


The most relevant previous works are those of Wolfe and Olhede [59], Chatterjee [24] and [30]. We 
provide comparisons for two types of bounded graphons: (1) /c-block graphons, and (2) a-Holder 
graphons. 

/c-block graphons. A /c-block graphon is a function on [0, l] 2 that is constant on rectangles of 
the form R x Ij, where Ji, form a partition of the interval [0,1]. 

In this setting, the oracle error e^\w) = 0 and e n {W) = Op(y/k/n) (see Appendix D). Our 
nonprivate estimator then has asymptotic error yj^ + A>"; this is dominated by e n (W) as 

long as k < pn and p > For constant e, our private estimator has asymptotic error at most 

\/ n + For ^ — (^/log 2 n) 1 / 3 and density p > the error of our estimator is 

again dominated by the (unavoidable) error of e n (W) = \fkjn. 

Several works analyze procedures for estimating the edge-probability matrix Q assuming that 
it is (exactly) a /c-block matrix. In the dense case p = 0(1), Gao et al. [301 , Theorem 1.1] show that 

the least squares estimator achieves error \\j } Q—^Q \\2 = Op(£) + They also give a matching 

lower bound, which shows that the MLE is optimal with respect to £2 estimation of Q. Chatterjee 
m Theorem 2.3] gives a polynomial-time algorithm with higher error | A} — ^Q ||2 = Op (■ 
These bounds apply to estimating the edge-probability matrix Q , but do not apply directly to 
estimating an underlying block graphon W. Lemma 14 shows that ^Q converges to W in the 62 


metric at a rate of Op{^/k/n). Using either of the algorithms above for estimating W gives a net 
error rate of Op(y/k/n) for 82 estimation of W. This is the best known nonprivate rate, and is 
matched by our nonprivate rate. 

In the sparse case, where p —> 0 as n —> 00 , Wolfe and Olhede showed under additional 
assumptions (roughly, that entries of Q are bounded above and below by multiples of p) that the 

MLE produces an estimate Q of Q that satisfies || = Op ^ + ^/ log 

H| Again, 


to get a rate oi f y/^^ + 

y/ log f or estimating an underlying /c-block graphon W. Note that when p is small, any 

of these three terms may dominate the rate. 


59l Theorem 5.11 


one can combine these with Lemma 


14 


Holder-continuous graphons. The known algorithms for estimating continuous graphons pro¬ 
ceed by fitting a /c-block model to the observed data, and arguing that this model approximates 
the underlying graphon. 

Our results show that if e is constant and W is a-Holder continuous (Lipschitz continuity 
corresponds to a = 1), then the nonprivate error scales as 2< * +1 + + ri ~ a ^ 2 for an 

appropriate choice of /c, while the private error scales as _|_ n ~ a / 2 for an 

appropriate choice of k. See Remark [2] for details. 

1 The guarantee in on Theorem 5.1] is given in terms of KL divergence. One can convert to £2 using the fact that 

D(p\\q) = ©((5 — p) 2 /p) when q — p is small relative to p. 


28 




























In the dense case (p = 12(1)), [30] show that one can estimate a ct-Holder continuous graphon 
by a fc-block graphon with error 


W W LS ) = Op(l + + k~ a + n-“/ 2 ), 


with the last term accounting for the difference between estimating Q and W. Setting k to the 
optimal value of k = l/n Q+1 gives a rate which except for the case a = 1 is dominated by the 
term e n (W) = 0(n _a / 2 ). Our nonprivate bound matches this bound for a < 1/2, and is worth for 
a > 1/2, while the private one is always worth. 

Wolfe and Olhede [59] analyse the MLE in the sparse case, again restricting to fc-block models. 
They shov0 


S 2 (W, Wmle) = Op 


log(n) . 4/log 2 * (l/p) log(fc) . h - a y/\og(n/p) 

p + V np + K + n«/4 


( 21 ) 


The value of k that maximizes this expression (asymptotically) can be found by setting the first 


and third terms to be equal; we get k = 
h (W 7 , Wmle ) = Op 


(ny/p/ log 


n 


i 

Q + l 


and a resulting bound of 


a. - - , - 

1 _ / log n / “+ 1 4/ log 2 (1/p) log(n) _J_ ylog (n/p) 


np 


+ 


y./4 


Next, note that p must be 12(log 3 (n)/n) for the middle term to be less than 1. This means that 

— a 

the first term is 0(n 2 (“ +1) ). For a < 1, this is never larger than the third term. We may therefore 
simplify the bound to Op f ^/ log ^/p) Io s(») _|_ \ ^ ag s t a ted in the introduction. 


B Background on Differential Privacy 

The notion of node-privacy defined in Section [2,3| “composes” well, in the sense that privacy is pre¬ 
served (albeit with slowly degrading parameters) even when the adversary gets to see the outcome 
of an adaptively chosen sequence of differentially private algorithms run on the same data set. 

Lemma 9 (Composition, post-processing [491126] ). If an algorithm A runs t randomized algorithms 
Ai,..., At, each of which is e-differentially private, and applies an arbitrary (randomized) algorithm 
g to their results, i.e., A{G) = g(Ai(G),... ,At(G)), then A is te-differentially private. This holds 
even if for each i > 1 , A{ is selected adaptively based on A\(G), ... ,4i_i(G). 

Output Perturbation. One common method for obtaining efficient differentially private algo¬ 
rithms for approximating real-valued functions is based on adding a small amount of random noise 
to the true answer. A Laplace random variable with mean 0 and standard deviation \/2X has 
density h(z) = 2 ^e _ l 2 l/ A . We denote it by Lap(A). 

In the most basic framework for achieving differential privacy, Laplace noise is scaled according 
to the global sensitivity of the desired statistic /. This technique extends directly to graphs as long 

2 We state [591 Theorem 3.1] for the special case where one searches over fc-block graphons in which all intervals 

have size 0(fe/n) (since allowing nonuniformly sized blocks only makes their bounds worse), the original graphon 

takes values in a range [A a , \b] defined by two constants such that 0 < A a < \b, and k is polynomially smaller than 

n. 
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as we measure sensitivity with respect to the metric used in the definition of the corresponding 
variant of differential privacy. Below, we explain this (standard) framework in terms of node privacy. 
Let G denote the set of all graphs. 

Definition 2 (Global Sensitivity [28]L The l\-global node sensitivity of a function f : G —> M p is: 

A/= max ||/(G)-/(G")||i. 

G,G node neighbors 

For example, the edge density of an n-node graph has node sensitivity 2/n, since adding or 
deleting a node and its adjacent edges can add or remove at most n — 1 edges. In contrast, the 
number of nodes in a graph has node sensitivity 1. 

Lemma 10 (Laplace Mechanism [28]). The algorithm A(G) = f(G) + Lap(A//e) p (which adds 
i.i.d. noise Lap(A//e) to each entry of f(G)) is e-node-private. 

Thus, we can release the number of nodes, v(G), in a graph G with noise of expected magnitude 
1/e while satisfying node differential privacy. Given a public bound n on v(G), we can release the 
number of edges, e(G), with additive noise of expected magnitude n/e. 


Exponential Mechanism. Sensitivity plays a crucial role in another basic design tool for dif¬ 
ferentially private algorithms, called the exponential mechanism. 

Suppose we are given a collection of Q functions, < 71 , .., qQ from G n to M, each with sensitivity 
at most A. The exponential mechanism, due to McSherry and Talwar m, takes a data set (in our 
case, a graph G) and aims to output the index i* of a function in the collection which has nearly 
maximal value at G, that is, such that q*(G ) ~ max* qi(G). The algorithm A samples an index i 
such that 

Pr(M(G) = i) oc exp (t^%(G)) • 

Lemma 11 (Exponential Mechanism |50], see also [27, Sec. 3.4]). The algorithm A is e-differentially 
private. Moreover, with probability at least 1 — 77 , its output i* satisfies 


qi* (G) > max (qi(G)) 

i 


2 A In (Q/g) 
e 


Lipschitz Extensions. There are cases (and we will encounter them in this paper), where the 
sensitivity of a function can only be guaranteed to be low if the graph in question has sufficiently 
low degrees. In this situation, it is useful to consider extensions of these functions from graphs 
obeying a certain degree bound to those without this restriction. 

Definition 3 ( G n 4 and vertex extensions). Let G Ut d denote the set of graphs with degree at most 
d. Given functions f : G n ^ —> M and f : G n —»• M, we say f is a vertex Lipschitz extension of f 
from G U) d to G n if f agrees with f on G and f has the same node-sensitivity as f, that is 

sup |/(G) - /(GO | = sup | f(G) - /(GO| . 

G ,G^ €.&n 

vertex neighbors vertex neighbors 

We close this section with the proof of Lemma [2] 
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Proof of Lemma [S| The existence of / follows from a very general result (e.g., |48L I42J ). which states 
that for any metric spaces X and Y such that Y C X, and any Lipschitz function f :Y —> M, there 
exists an extension / : X —> M with the same Lipschitz constant. The explicit, efficient construction 
of extensions for linear functions is due to Kasiviswanathan et al. m- The idea is to replace f(G) 
with the maximum of /(C) where C ranges over weighted subgraphs of G with (weighted) degree 
at most d. It is the value of the following linear program: 


/(G) = max/(C) such that 


C E [0, l] nxn is symmetric, and 
< Cij < A(G)ij for all i,j, and 
, zLj/* Ci ,3 < d for all i E [n\. 


See [41] for the analysis of this program’s properties. 


□ 


C Auxiliary Bounds on Densities and Degrees 

Lemma 12. Let W : [0, 1] 2 — > [0, A] be a normalized graphon, let p E (0,A _1 |, let Q = H n (pW), 
let G = G n (pW) and assume that pn is bounded away from zero. Then 

1. Ep(Q) = E p(G) = p, Var(p(Q)) = 0{p 2 A/n) and Var(p(G)) = 0(p 2 A/n), so in particular 

Pr{|p(G) -p\> 5p} = O(^) and Pr{|p(Q) - p\ > dp} = o(J^j . 
for any 6 > 0. 

2. Let e n (W) = \\W - W[Q] || 2 . Then 

(1 - tn(W))— <d91< (1 +£ „(J V)) — 

n — 1 p n — 1 


3. Let 5 E 0 < 5 < 1. With probability at least 1 — 2e s' 5 p(Q) n ; 


1 — <5 < 


P(G) 

p(Q) 


<1 + 5. 


Proof. 1. Clearly, Ep(Q) = Ep(G) = p. 

To bound the variance of p(Q), we expand fy Var(p(Q)) as a sum of n 2 (n — l) 2 /4 terms of the 
form E \W(xi,Xj)W{xk,Xi)\ — E[W(xj, Xj)]K[W(xk, xf)\ with i < j and k < l. Observing that only 
those terms contribute for which either i = k, j = i or j = k, and boundingE [W (, Xi , Xj)W(xk, xe)} < 
||WHl < HWlloollWlli, we obtain that the variance of ^p(Q) is 0(A/n). 

To bound the variance of p(G), we first condition on X = (x\, ..., x n ), and bound 

E[p 2 (G) I X] = P HQ) + n2( /_ 1)2 y (Qij - Qfj) < p\Q) + XL T] p(Q). 


Taking the expectation over X and using the bound on the variance of p(Q), we obtain that 

'p 2 A p \ /p 2 Ai 


V ar(p(G)) = Of?- + 4) = o (?—). 

V n n z / V n / 
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where in the last step we used the assumption that pn is bounded away from zero. 

2. Note that p(Q) = ^rpllQUi- Next, we use the triangle inequality and the fact that the 
Li-norm is bounded by the L 2 - n orm to see that 


3. 

E(S) 

0<1 


IIQIb 


-1 


IIQIb 


< || -W[Q] - W||i < || -W[Q\ - W\\ 2 = e n (W) 


Conditioned on X, S = ^ p(G) is a sum of Bernouilli random variables with mean 

p(Q). By the multiplicative Chernov bound from Lemma 


_ n(n— 1 ) 
— 2 


17 


we have that for all 


Pv{\p(G)-p(Q)\>tp(Q)\x}< 2 exp^— 


6 2 n(n — 1) 


~P 


< 2 exp- n p 

6 


□ 


Next we bound the maximal degree in G = G n (pW). 

Lemma 13. Let W be a normalized graphon with HWHoo < A, let 0 < pA < 1, and let /3 > 1. 
Then with probability at least 1 — nexp(—(3Apn/3), the maximal degree in G n (pW) is bounded by 
(1 + /3)Apn. 

Proof. Note that the degrees in G n are stochastically dominated by those in an Erdos-Renyi random 
graph where edges are chosen i.i.e. with probability pA. In such a graph, the degree of a given 
vertex i is a sum of n — 1 i.i.d Bernouilli random variables, and the standard Chernov bound implies 
that for all (5 > 1 

Pr(dj > npA(l + /?)) < exp^—^npA^J. 

Taking the union bound over all n vertices in G n (pW) this proves the claim. □ 

D Convergence of the edge-probability matrix for /c-block graphons 

The sampling error e n (W) plays a key role in the statements of Theorems [2] and [l] In some settings 
such as Holder-continuous graphons, this sampling error is dominated by the oracle error e^\w). 
For /c-block graphons, however, e'P\w) = 0 and e n (W) becomes more significant. 

Lemma 14. IfW is a k-block graphon, thenE(e n (W) 2 ) = 0(A 2 yjk/n) andE(e n (W)) = 0(Af/k/n) 
where A = || W||oo- 

Proof. Fix a /c-block graphon W, and let pi, ...,Pk be the lengths of the “blocks”, that is the intervals 
defining the block representation (so that pt > 0 and YltPt = !)• Given a sample x\, ...,x n of i.i.d. 
uniform values in [0,1], let pt denote the fraction of the x* that land in each block t. 

Aligning Q = {W{xi,Xj )) i with W consists of finding a permutation 7r of [n]. This maps 

each Xi to one of the intervals I\, ,.,I n where Ig = pj- 

We say Xi is correctly aligned if its interval is contained in the block in which Xj landed. 
For each block t, we can ensure that nmm{p t ,pt} — 2 of the points Xi that landed in t get aligned 
with t (the —2 term accounts for the fact that up to 1/n of the length at each end of the interval 
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does not line up exactly with one of the intervals If). Thus, the number of points Xi that get 
incorrectly aligned is at most n^Zt ( Pt ~ min {pt,Pt} + 2/n) = ^|| p — p\\i + 2k. 

Each misaligned point contributes at most 2A 2 /n to the total squared error \\W — Q\\ 2 , so we 
have 

6 2 2 (W,Q)<A 2 (\\p-p \\ 1 + 4k/n). 

Each term \pt~Pt\ in the £1 norm on the right-hand side is the deviation of a binomial from it’s mean, 
and has standard deviation y/pt(l — Pt)/n. This upper bounds the expected absolute deviation by 
Jensen’s inequality. Thus, E (S 2 (W, Q )) < A 2 VPtJ n+AA 2 k/n. The sum Et \/Pt/ n is maximized 
when pt = 1 /k for all f; it then takes the value k^/lj~(kn) = yjk/n. Hence E(5 2 (W, Q)) < 
A 2 \/ k / n + AA 2 k / n < hK 2 ^Jk/n. By Jensen’s inequality, E(<5 2 (W,Q)) < y/5Af/k/n , as desired. □ 

Corollary 1. For any graphon W, we have E(e n (lT)) < 2e ( j®\w) + 0{y/k/n). 

Proof. Fix a matrix W, and let Wp denote the best fc-block approximation to W in the L2 norm 
(that is, the minimizer of e'{f > \w)). Given a uniform i.i.d. sample in x\, ...,x n , let H n (W) denote 
the matrix (W(xi,Xj)) i 1 , and let H n (Wp) denote (Wp(xi, Xj)) t j £ [ n y By the triangle inequality, 


t (lT) = 5 2 (W, H n ) <\\W- W P || 2 +e n (W P ) + || H n (W) - H n {W P )|| 2 . 


1 °) 


(W) 


Lemma 14 bounds e n ( Wp ) by f/k/n. It remains to bound the last term. Squaring it, we have 
\\H„,(W) — H n {W P )\\ 2 = Yhi< j |(W — Wp)(xi, Xj)\ 2 . These terms are not independent, but each 
individually has expectation \\W — Hp|| 2 = e[,°^(lT) 2 . By linearity of expectation, E\\H n (W) — 
H n (Wp )|| 2 < e[ 0) (W) 2 , and hence E|| H n (W) - H n (W P )|| 2 < ef\w). □ 


E Bounds for Holder-Continuous Graphons 

In this section we prove Remark [ 2 ] Throughout this section we assume that W : [0, l] 2 —> [0,1] is 
a-Holder continuous for some a £ (0,1], i.e., we assume that there exists a constant C < 00 such 
that 

I W(x,y) - W{x',y’)\ < c(\x - x'\ + \y - y'l') . 

Lemma 15. Let H = H n (W), and assume that the vertices of H are reordered in such a way that 
x\ < x 2 < ■ ■ ■, x n . Then 

\\W-W[H n ]\\ 2 = 0 P {n- a ' 2 ). 

Proof. We first approximate W in terms of the weighted graph H with weights (H)ij = W(xi,xj), 
where x* = is the expectation of x\. Since \x — xf < ^ when x £ [^, ^], we can use the 
Holder continuity of W to conclude that 

\\W - W[H ]|| 2 < \\W - W[H ]||oo < C'(^) a - 
To prove the lemma, it is therefore enough to prove that 


E 


\\W[H}-W[H]\\ 2 =0(n~ 
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where E[-] denotes expectations with respect to the random variables x\,... ,x n . Using the Holder 
continuity of W together with Jensen’s inequality, we bound 


E 


\\W[H n ]-W[H]\\l\ = -? J2 ^[{Wixi.xj) -W{x uXj )Y 


i,je[n] 


< c 4 


Y E (\Xi - Xi\ + \ X j - Xj\) 

i,j£[n] 


2a 


C 2 


n* 


— r,2 Yj ^ \ Xi Xi \ + 2 I Xj Xj | ) 

+ 2E 


«J6|« J 

<-E(® 

i,je[n] 


n 2 


\Xi Xi\ 


\Xj-Xj I 


Using the fact that x* has expectation X* = ^-j- and variance ^A_Xj(l — xf) < we bound the 
right hand side by C 2 n~ a , completing the proof. □ 

Lemma 16. Let Vk be a partition of [0,1] into adjacent intervals of lengths l/k. Then 

cf\w) < \\W-W V J X < c(fy. 

Proof. Let Vk = For (x,y) G p x Ij, W-p k (x,y) is an average over points in p x /j, 

implying that \W(x,y) — W-p k (x,y)\ < C(2/k) a . □ 


F Consistency of Multi-way cuts 

In this appendix, we prove the following theorem which implies Theorem [3] by the same arguments 
as those which lead from Theorems [5] and [4] to Theorems [T] and [2j 

Theorem 6. Let q > 2 be an integer. 

(i) Under the assumptions of Theorem [I| 

d^ aus (S q (G), S q (B nonprivate )) < 2ef\H n (W)) + O p \ + • ( 22 ) 

(ii) Under the assumptions of Theorem ^! J 

d^ aus (S q (G), S q {B private )) < 2e[ °\H n (W)) + O p (J + A^ ■ (23) 

Before we prove the theorem, we start with a few bounds on the Hausdorff distance of various 
sets of q- way cuts. First, using the definition of the cut-distance (and the fact that the set of q- way 
cuts of a graph is invariant under relabelings), it is easy to see (see also [IB] ) that whenever G and 
G' are weighted graphs on [n] and V is a partition of [n], then 

\\a/V - G'/V\\ x < pr^G') 
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implying that 

d^ us (S q (G), S q (G')) < <fa(p^G, (24) 

In a similar way, we have that for two graphons W, W', 

d£-(S,(W),S,(W’)) < fa(ph-IV, — F-W'). (25) 

We will also need to compare the fractional and integer cuts, S g (G ) and S q (G'). To do so, one can 
use a simple rounding argument, as in Theorem 5.4 and its proof from [16) . This gives the bouncfi] 

d% w (S g (G),S q (G)) < (26) 

l|Gpv n 

valid for any weighted graph G with node weights 1 on [n] and maximal edge-weight 1. We also 
note that for any weighted graph Q, 


S g (Q) = S q {W[Q ]) 


(27) 


(see [16, Proposition 5.3] |^j). Finally, we note that 


||G||i-IIG'lli <S a (G,G') and 


i — ||VF'||i| < ||W — VF'lln. In particular, 

i-||W"||i < 8 u(W,W'). (28) 


Proof of Theorem [h| Let Q = H n (pW) and G = G(Q). By Lemma 12, we have that p(Q) E 
[p/2,2p] with probability at least 1 — 0(A/n). By the assumptions of the two theorems, pn > 2 log 2. 

We apply mi Lemma 7.2] to show that <5n(G, Q ) = pO ^) with probability at least 1 — 0(A/n). 
As a consequence, again with probability 1 — 0(A/n), 




1 


Q, 


l 


IIQIIi IIQII 


G =0 J — 



pn 


A P 


we 


By (24) and (28), this implies that with the same probability 

<e-(S,(G).S,(0)) = o(y/l). 

Next we apply (26) to the weighted graph Q' = Since mq 1 ,ii = < 

conclude that with probability at least 1 — 0(A/n), 

<i"““(s,(<3),4(<3)) = <£*”*(«,«), s,(«')) = o(JP + Ay 

' V U f v 4/ / t / 

Since ||F||oo < 1 for all F E S q (G) and all F E S q (Q), we can easily absorb the failure event, getting 

d’^r(s,(o),s,m = o P (^F+F). ( 29 ) 


d To translate the results from [ 16 ] into ( 26 ), we need to take into account that in | 16 | , quotients where defined 
with a normalization of Tj- instead of „a hgHj^ (leading to the factor pip- on the right hand side of (261), and that 
Hausdorff distances were defined with respect to the Li-norm (leading to a bound which is better by a factor q than 
the bounds in USD- 

4 Note that in : ;l6] the notation for integer and fractional partitions is the reverse of the one used here and in [ 18 ] . 
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To complete the proof, we proceed as in the proof of (14) to show that 


Combined with the bound from Theorem [2j the fact that the cut-norm is bounded by the L 2 -norm, 
and the fact that fa (H n (W), < |1 — p(Q)/p\ = Op(A/n), we conclude that 


fa («4 Q ' W) w ^) £ 4 0) (ffn m+o P U »(£ 4- logt 


n 2 p 


np 


Combined with (27), 


identical, except that now we use Theorem [l] 


(28), and the bound (|29|) , this proves (22). The proof of (23) is essentially 

□ 


G Useful Lemmas 

Lemma 17 (Multiplicative Chernoff bound). Let X\, X 2 ,..., X n be independent random variables 
taking values in [0,1], and let X = La=i U- If E(X) < po and fd < 1, then 

Pr(|X - E(X)| > Ppo) < 2 exp(-^Vo/3). 

For (d > 1, the probability is at most 

Pr(|X - E(X)| > Pp 0 ) < 2exp(-/3/i 0 /3). 

Proof. Let p = E(X) denotes the exact mean of X (so p < po). The standard multiplicative form 
of the Chernoff bound states that for 5 > 0 (not necessarily less than 1), we have 

Pr(|X - EX| > dp) < 2 max(e _ 5 <s2/i , e~^). 


a _lPfao _lo 

Setting Sp = /dpo (that is, 5 = the bound above becomes 2max(e 3 ^ ,e 3 W). Both of 
these terms are bounded above by 2exp(—f3 2 po/3): the first, since po < //; and the second, since 


/ 3 < 1 . 


□ 
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